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ABSTRACT 

This paper departs from the usual subject of the Space Simulation Conference 
and concerns itself with the need for testing under simulated mission oper- 
ational conditions and reviews the results of such tests from the point of 
view of the user. It presents a brief overview of the usual test sequences 
for high reliability, long life spacecraft and will analyze the effectiveness 
of the testing program in terms of the defects which are discovered by such 
tests. The need for automation, innovative mechanical test procedures and 
design for testability will be discussed. 

INTRODUCTION 

As our spacecraft have become progressively more complex and their 
missions have become more sophisticated and of longer duration, it has be- 
come increasingly important to improve the methods for verifying design and 
workmanship prior to making the decision to launch. Such verification is 
based on an extensive testing program which duplicates operation over a range 
of worst case mission scenarios and simulates exposure to launch, boost and 
space flight environments. 

Since a typical spacecraft may cost $40 million, contain 60,000 
electronic piece parts and must last 7 to 10 years in orbit, it clearly is 
wise to learn as much as possible about how well it can be expected to per- 
form in space. A comprehensive test program must progress in a logical 
manner from development testing, through qualification and acceptance 
testing to flight readiness checkout at the launch site. In addition, the 
test program must include development, qualification and acceptance tests at 
progressively higher levels of assembly starting with parts and continuing 
th’X)ugh unit level (i.e., component, black box or assembly) to subsystem, 
spacecraft and system level tests. The real challenge lies in choosing the 
most perceptive and cost effective test procedures and environmental exposures 
at each assembly level. These must be tailored to the specific spacecraft 
and mission at hand but can be summarized for purposes of this paper as 
described below. 


PARTS TESTING 

Current procedures for piece-part testing are based on a large body 
of engineering data and various analyses of the physics of failure for 
the different kinds of parts. Standardization of test procedures has been 
fostered by the government and the electronic parts industry. For example, 
MIL-STD-883 defines screening tests for micro-electronic devices and is 
intended to yield an in-equipment failure rate of less than .004% per 
thousand hours for high reliability (Class S) parts. Detailed screening, 
sampling and lot qualification tests are specified, including temperature 
cycling, burn-in, particle impact noise detection (FIND), radiographic 
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inspection and life tests, among others. Comparable specifications and 
procurement standards exist for other electronic parts and are reflected 
in the design manuals and procedures used by individual aerospace contractors. 
These procedures have resulted in a significant drop in part failures during 
subsequent unit level testing. 


COMPONENT TESTS 

Unit or component- level test procedures are designed to verify all 
functional performance requirements over a range of environmental exposures 
without disassembly or change in the configuration of the component as 
installed in the spacecraft. For electronic components this usually 
requires that test points be brought out to a special test connector; this 
makes it possible to inject test signals and observe the resulting 
response, waveforms, logic levels, etc. Computer controlled special test 
equipment is required for functional checkout of digital components and 
complicated electronic components which have several modes of operation 
and process a variety of signals. Mechanical and electromechanical 
components also require special purpose test equipment and test fixtures 
for functional checkout but generally do not require automated testing. 

The environmental tests for component- level qualification and acceptance 
are specified in MIL-STD-1540A "Test Requirements for Space Vehicles". The 
newer spacecraft projects, especially those for important military missions, 
use the approach spelled out by this document. It involves functional testing 
before, during and after exposure to pyroshock, random vibration or acoustics, 
thermal cycling, vacuum and EMI. Depending on the nature of the component, 
thermal cycling is performed at ambient pressure or in a vacuum chamber. In 
either case at least eight temperature cycles should be applied to expose 
any workmanship or design defects before acceptance. Test levels and duration 
vary from unit to unit but should be selected to accelerate detection of 
inherent defects without inducing damage or degradation of good equipment. 

SPACECRAFT TESTS 

A simplified version of a typical spacecraft-level test flow is shown 
in Figure 1. There are three separate phases, namely an integration phase, 
an environmental qualification or acceptance phase and a prelaunch verifi- 
cation phase. During the integration phase the various components, 
subassemblies and harnesses which have undergone testing at the component and 
part level are installed on the spacecraft structure and interconnected to 
form subsystems and related equipment groups. Physical inspection and 
functional tests are performed under ambient conditions to verify interfaces 
and correctness of the assembly procedures. The environmental qualification 
or acceptance phase consists of functional performance tests under various 
environmental conditions to demonstrate that the full range of performance 
requirements is met. The usual approach is to conduct end-to-end tests 
which closely simulate the actual in-orbit mission. Large deployable 
appendages such as solar arrays and antennas are usually tested separately 
and may be removed during various electrical functional tests. The prelaunch 
verification tests are performed at the launch site and vary greatly from 
program to program. On some spacecraft a full functional test is performed 
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prior to installing the spccecraft on the booster. In other cases only 
on-stand interface and comr.iand compatibility tests are conducted in addition 
to ordnance circuit checkout and fueling operations. 



igure 1 - Spacecraft Test Flow 
EVALUATION OF TEST RESULTS 

The ultimate objective is to launch spacecraft which will operate 
flawlessly in orbit for their specified lifetime or longer. We therefore 
strive for failure-free acceptance testing under conditions which closely 
simulate orbital conditions. Discrepancies which are discovered during 
acceptance testing are corrected prior to launch. In Figures 2 and 3 we 
have attempted to correlate orbital performance to perfoiTtiance during 
acceptance testing prior to launch. The average number of on-orbit and 
acceptance test discrepancies for four different projects representing a 
total of 25 spacecraft was normalized with respect to spacecraft complexity 
as measured in terms of part count and the average number of defects per 
1000 parts was plotted for each project. Note that lower defect rates 
during acceptance testing yield better on-orbit performance. This is 
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also borne ojt by Figure 3 which plots normalized on-orbit discrepancies 
against spacecraft level acceptance test discrepancies fo 12 individual 
spacecraft comprising three separate projects. 



Figure 2 - Orbital Performance Correlation by Project 


ANALYSIS OF TEST PROGRAM 

In order to determine whether the current approach of duplicating antici- 
pated operating modes and simulating space environments during the qualifica- 
tion and acceptance test program is really effective in locating all potential 
defects, we reviewed the actual test history for two recent spacecraft projects 
in more detail. Tables 1 ar 2 give the number and type of discrepancies 
which were found during successive phases of the testing program and during 
orbital operation. Project A consists of a qualification test spacecraft and 
three identical flight spacecraft all of which are currently operating in 
orbit. Each satellite contains approximately 58,000 electronic parts 
including about 5500 integrated circuits. Project B consists of three 
scientific satellites, all of which have been launched. The first of the 
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three spacecraft served as both a prototype and flight vehicle since there 
wes no separate qual spacecraft. Not including the scientific payload, each 
satellite contains approximately 30,000 electronic parts. The data of Table 2 
does not include discrepancies found in the scientific instruments payloads 
since these were GFE. 
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Figure 3 - Orbital Performance vs. System Test Results 


Both Tables 1 and 2 provide the number of test discrepancies due to 
defects in the equipment under test, due to defects in test procedures and 
due to causes which are net related to any defect. Note that 367 and 174 
product defects respectively were found during component level qualification 
and acceptance testing which were corrected prior to start of spacecraft level 
testing. Additional component defects were found during spacecraft level 
testing in the factory and at the launch site and during on-orbit operation. 
These additional discrepancies are called "escapes" because they escaped 
detection during component qual or acceptance testing. The escape ratio Ec 
is simply the ratio of component test escapes to total component defects. 
Similarly, the on-orbit discrepancies or performance anomalies are escapes 
from the ground test program if it is assumed that there are no spontaneous 
failures due to wearout or other causes during the mission life of the space- 
craft. The escape ratio Eq then is the ratio of on-orbit discrepancies to 
total discrepancies. The purpose of these ratios and the data of Tables 1 
and 2 is to provide a broad overview of what results can be expected from 
typical testing programs. 
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Table 1. PROJECT A - CAUSE OF TEST DISCREPANCIES 




test 

LEVEL 


ESCAPE ratio"! 

DEFECT /CAUSE OF DISCREPANCY 

COMPONENT 

SPACECRAFT 

LAUNCH BASE 

ON ORBIT 



PRODUCT DEFECTS 







PART FAILURE 

68 

10 

1 


U,9 


COMPONENT MANUFACTURING 

112 

19 


1 

14.5 

0.7 

COMPONENT DESIGN 

174 

28 



13.8 


SPACECRAFT ASSEMBLY 


14 





SPACECRAFT DESIGN 


16 

1 

3 


15 

TEST INDUCED 

13 

5 





UNDETERHINEO 


n 





TOTAL 

367 

103 

2 

4 

14.3* 

o.m 

PROCEDURE DEFECTS 







DOCUMENTATION 

no 

34 


j 



TEST EQUIPMENT 

41 

23 

1 




OPERATOR ERROR 1 

35 

16 

1 

j 

1 


TEST SETUP 

18’ 

3 

1 

. 




TOTAL 

204 

76 

1 

3 : 




NON-DEFECTS 







WITHIN TOLERANCE/WAIVER 

78 

21 

2 

4 



UNABLE TO REPEAT PROBLEM 

29 

13 


1 



TOTAL 

107 

34 

2 

6 




NOTES: 4 spacecraft, 3 in orbit, 3 years of orbital performance 


To gain further insight into the effectiveness of environnental simulation 
for spacecraft level testing we have broken down the product defect discre- 
pancies as shown in Tables 3 and 4 for Projects A and B respectively. Note 
that in both projects about 40% of the spacecraft acceptance test defects 
were found during the integration phase (Figure 1). Project A spacecraft 
received a temperature cycling test in a thermal chamber at ambient pressure 
during which 15 discrepancies were observed. During the subsequent T/V test 
four component failures occurred, all on the qualification spacecraft. These 
failures most likely were not induced by the vacuum environment since each of 
the failed components had previously passed a T/V test as part of the compo- 
nent acceptance sequence. Also note that on Project B only one component 
discrepancy was found during spacecraft T/V whosj discovery cannot be credited 
to vacuum exposure. It appears therefore that exposure to vacuum as part of 
spacecraft level acceptance testing is not as profitable as testing over the 
widest possible temperature range in a thermal chamber. The merit of temper- 
ature cycling at the spacecraft level has also been demonstrated on a classi- 
fied project as described in Reference 1. 

Careful analysis of the discrepancy data for Projects A and B and several 
other projects leads to the following additional conclusions: 

a) A majority of on-orbit discrepancies are due to subtle design 
defects which wei'e not or could not be discovered during the 
qualification and acceptance test program. 

b) Repetitive room ambient testing over a variety of mission scenarios 
and using different test methods will disclose more defects than 
simple environment?^ exposure without electrical stress. 
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Although it is impossible to achieve reliable operation in orbit by testing 
alone, it seems axiomatic that the more testing and the more variation in 
testing which is conducted, the better the orbital performance. This 
justifies the use of automated test equipment and requires design of space- 
craft for testability as described below. 


Table 2. PROJECT B - CAUSE OF TEST DISCREPANCIES 


DEFECT/CAUSE OF DISCREPANCY 

~~ test level I 1 

ESCAPE 


COMPONENT 

SPACECRAH 

LAUNCH BASE 

ON ORBIT 



PRODUCT DEFECTS 





m 


FART FAILURE 

38 

4 


1 


2.3 

COHPONENT MANUFACTURING 

79 

5 

1 




COMPONENT DESIGN 

45 

19 

4 

5 

38.3 

6.8 

SPACECRAFT ASSEFBLV 

- 

18 





SPACECRAFT DESIGN 


11 

2 

1 



TEST INDUCED 

6 


2 




UNDETERMINED 

6 

4 

1 




total 

174 

61 

11 

•t 

19.8 

2.9% 

PROCEDURE DEFECTS 







DOCUMENTATION 

55 

23 

1 




TEST EQUIPMENT 

62 

4 

1 




OPERATOR ERROR 

50 

13 





TEST SETUP 

35 

T 

3 




TOTAL 

202 

47 

5 




NON-DEFECTS 







WITHIN TOLERANCE/WAIVER 

21 

21 

2 




UNABLE TO REPEAf PROBLEM 

35 1 

7 

1 

1 



T0T4L 

56 : 

1 

28 

3 

1 




NOTES: spacecraft » 3 on orbit, 4 years of orbital performance 

Experiment TDRs not included 


AUTOMATED TESTING 

An objective of the spacecraft acceptance test sequence is to exercise 
all equipment and verify all the operating modes which the spacecraft will 
encounter during its space mission. To get an idea of the complexity of such 
a comprehensive spacecraft test (CST) consider the Tracking and Data Relay 
Satellite (TORS) shown in Figure 4, which is currently under development at 
TRW. The TDRS system provides two-way communications with a large ground 
station for up to 20 spacecraft users in low earth orbits including the 
shuttle orbiter. In addition, it provides twelve 18 Wiz repeater channels 
at C band for commercial communication satellite service. Altogether, there 
are more than 60 switchable RF conmunications links. Each TDRS contains 172 
active electronic units which contain approximately 55,000 parts of which 
4500 are integrated circuits. One thousand seventy-six discrete and 58 serial 
ground commands are available for configuration control, redundancy switching 
and power management during on-orbit operation. One thousand two hundred 
eighty-eight performance and status parameters are telemetered to the ground. 
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Table 3. PROJECT A - LOCATION OF TEST DISCREPANCIES 



ELECTRONIC COMPONENTS 1 

PROPULSION/STRUCTURE 

HARNESS 


lEST/DEFECT 

I 

Part 

Workman- 

ship 

Des 1 gn 

Part 

Workman- 

ship 

Design 

Workman- 

ship 

Design 

TOTAL 

DEFECTS 

UNIT ACCEPTANCE 

68 

100 

167 


12 

7 



354 

SPACECRAFT 

ACCEPTANCE 

•11 

15 

25 

2 

23 

2 

7 

2 

87 

Integration 

(3) 

(6) 

(12) 

(1) 

(11) 

(1) 

(3) 

(1) 

(38) 

First Functional 

(3) 

(2) 

(5) 






(10) 

Temperature 

(2) 

(21 

(7) 


(1) 


(2) 

(1) 

(15) 

EMC/IK 

(1) 

(1) 

(1) 


(4) 


(1) 


(S) 

Dynamics 
Post Dynamics 





(4) 


(1) 


(5) 

T/V Test 

(2) 

(2) 







(4) 

Preship 

Functional 


(2) 



(3) 

(1) 



(6) 

Launch Base 




(1) 





(1) 

ON ORBIT 



1 


1 

C 
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Table 4. PROJECT B - LOCATION OF TEST DISCREPANCIES 



electronic components I 

PROPULSION/STRUCTURE 

HARNESS 


TEST/OEFECT 

Part 

Workman- 

ship 

Design 

Part 

Workman- 

ship 

Design 

Workman- 

ship 

Design 

TOTAL 

DEFECTS 

UNIT ACCEPTANCE 

35 

79 

43 

3 


2 



162 

SPACECRAFT 

ACCEPIANCE 

5 

6 

23 


7 

1 

11 

12 

65 

Integration 



(9) 


(5) 

(1) 

(2) 

(9) 

(26) 

EMC 


(2) 

(1) 




(1) 

(1) 

(5) 

First Functional 

(1) 

(2) 

(4) 


(1) 


(3) 


(11) 

Dynamic 

Environment 

(1) 

(1) 

(1) 




(1) 


(4) 

Post Dynamic 
Functional 

(1) 


(2) 




(2) 


(5) 

T/V Environment 



(1) 






(1) 

Preship Functional 

(1) 


(1) 


(1) 


(2) 


(5) 

ETK 

(1) 

(1) 

(4) 





(2) 

(8) 

ON ORBIT 

1 


5 



1 



7 
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Figure 4. TORS Spacecraft Configuration 


Because of this complexity, a computer is required for generating command 
sequences, processing of telemetry data, conversion of RF measurements to 
performance parameters appearing in requirements specifications and for 
recording test results. In addition, a computer is needed to control the test 
equipment which supplies the RF signals, sensor stimuli and electrical power 
which are required for operation of the spacecraft. 

The significance of computer based automated testing must be recognized 
by program managers and test facility managers. Although computers with 
sufficient speed and memory capacity to handle almost any spacecraft system 
test requirement are readily available, the software needed for conducting 
the test and processing the data must be designed for the specific spacecraft 
and costs significantly more than the computer hardware. Standardized software 



and distributed processors are being used wherever possible to reduce this 
high cost, but a lot more effort and innovation is required to arrive at 
optimized automated test systems. 

TESTABILITY 

Another issue which is of major concern to system test organizations is 
the basic testability of complex spacecraft. Over the last 10 to 15 years we 
have seen a change in the characteristics of spacecraft so that today we have 
more and more digital equipment using LSI and on-board processing and much 
larger and more complicated mechanical structures. In addition, the cross- 
strapping and reconfiguration capability of the electronic equipment is so 
great that it is very difficult to validate all anticipated operating modes 
prior to flight. It is, therefore, very important for test people to partici- 
pate in the design of a spacecraft from the start in order to influence the 
following design features: 

e selection of teH points and location of test connectors 

• equipment layout and clearances to enable replacement of 
faulty units 

• provision of accelerated cc:.niand capability for automated 
testing 

• selection of telemetry measurements 

• design of on-board computers for reprogrammability and 
interfacing with ground test equipment 

• accessabil ity for x-ray and visual inspection of 
critical connectors 

• mechanical design which enables functional testing of mechanisms 

• location of hard points to allow safe support during testing 
and transportation 

• use of test-only sensors vs. flight telemetry 

Several of these points may appear trivial or obvious, but experience has 
shown that their importance must not be underestimated. Another observation 
which should influence future activities is the fact that through automation 
and improved test equipment we have kept pace with the increasing complexity 
of electronic and electrical equipment aboard a spacecraft. On the other 
hand, our mechanical and structural test procedures have not changed signifi- 
cantly during the last 10 years. Acoustic and vibration test facilities are 
available but methods for conducting deployment tests under simulated zero 
gravity conditions are still relatively crude. On-orbit testing using the 
shuttle as a base may prove to be a viable option in the future and deserves 
further study. 
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CONCLUSIONS AND RECOMMENDATIONS 


Performance during acceptance testing affects performance in orbit. 
Comprehensive testing over a wide range of usage and environmental conditions 
has been a major contributor to the excellent success record of spacecraft 
which have been launched. 

Severe thermal thermal cycling over the maximum allowable temperature 
range rather than extended thermal vacuum exposure should be used to detect 
workmanship and design defects at the spacecraft level. 

Test engineers and project managers must be aware of the large cost and 
schedule requirements for generating the software which controls the automated 
test procedures. 

Methods for testing the deployment of appendages and/or verifying 
mechanical characteristics of large structures will have to be improved as 
structures become larger and more flexible and dynamic interactions which 
influence alignment and pointing accuracies become more critical. 

Test engineers must participate in early spacecraft design decisions and 
exert greater influence to insure that testability is achieved. 
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